notebook.community

Edit and run 





In [1]:



    


import tensorflow as tf
import numpy as np
from tensorflow import data
import shutil
import math
from datetime import datetime
from tensorflow.python.feature_column import feature_column

print(tf.__version__)



    


















    






/Users/khalidsalama/anaconda/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
  return f(*args, **kwds)










    






1.4.0

Steps to use the TF Estimator (Train_And_Evaluate) APIs

  1. Define dataset metadata
  2. Define data input function to read the data from .tfrecord files + feature processing
  3. Create TF feature columns based on metadata + extended feature columns
  4. Define an estimator (LinearCombinedDNNClassifier) with the required feature columns (wide/deep) & parameters
  5. Run an experiment using the estimator train_and_evaluate function to train, evaluate, and export the model
  6. Evaluate the model using test data
  7. Perform predictions & serving the exported model




In [2]:



    


MODEL_NAME = 'class-model-01'

TRAIN_DATA_FILES_PATTERN = 'data/train-*.tfrecords'
VALID_DATA_FILES_PATTERN = 'data/valid-*.tfrecords'
TEST_DATA_FILES_PATTERN = 'data/test-*.tfrecords'

RESUME_TRAINING = False
PROCESS_FEATURES = True
EXTEND_FEATURE_COLUMNS = True
MULTI_THREADING = True

1. Define Dataset Metadata

  • tf.example feature names and defaults
  • Numeric and categorical feature names
  • Target feature name
  • Target feature labels
  • Unused features




In [3]:



    


HEADER = ['key','x','y','alpha','beta','target']
HEADER_DEFAULTS = [[0], [0.0], [0.0], ['NA'], ['NA'], [0.0]]

NUMERIC_FEATURE_NAMES = ['x', 'y']  

CATEGORICAL_FEATURE_NAMES_WITH_VOCABULARY = {'alpha':['ax01', 'ax02'], 'beta':['bx01', 'bx02']}
CATEGORICAL_FEATURE_NAMES = list(CATEGORICAL_FEATURE_NAMES_WITH_VOCABULARY.keys())

FEATURE_NAMES = NUMERIC_FEATURE_NAMES + CATEGORICAL_FEATURE_NAMES

TARGET_NAME = 'target'

TARGET_LABELS = ['positive', 'negative']

UNUSED_FEATURE_NAMES = list(set(HEADER) - set(FEATURE_NAMES) - {TARGET_NAME})

print("Header: {}".format(HEADER))
print("Numeric Features: {}".format(NUMERIC_FEATURE_NAMES))
print("Categorical Features: {}".format(CATEGORICAL_FEATURE_NAMES))
print("Target: {} - labels: {}".format(TARGET_NAME, TARGET_LABELS))
print("Unused Features: {}".format(UNUSED_FEATURE_NAMES))



    


















    






Header: ['key', 'x', 'y', 'alpha', 'beta', 'target']
Numeric Features: ['x', 'y']
Categorical Features: ['alpha', 'beta']
Target: target - labels: ['positive', 'negative']
Unused Features: ['key']

2. Define Data Input Function

  • Input .tfrecords files name pattern
  • Use TF Dataset APIs to read and process the data
  • Parse tf.exmaples to feature tensors
  • Apply feature processing
  • Return (features, target) tensors

a. Parsing and processing logic





In [4]:



    


def parse_tf_example(example_proto):

    feature_spec = {}

    for feature_name in NUMERIC_FEATURE_NAMES:
        feature_spec[feature_name] = tf.FixedLenFeature(shape=(), dtype=tf.float32)
    
    for feature_name in CATEGORICAL_FEATURE_NAMES:
        feature_spec[feature_name] = tf.FixedLenFeature(shape=(), dtype=tf.string)
    
    feature_spec[TARGET_NAME] = tf.FixedLenFeature(shape=(), dtype=tf.string)

    parsed_features = tf.parse_example(serialized=example_proto, features=feature_spec)
    
    target = parsed_features.pop(TARGET_NAME)
    
    return parsed_features, target


def process_features(features):

    features["x_2"] = tf.square(features['x'])
    features["y_2"] = tf.square(features['y'])
    features["xy"] = tf.multiply(features['x'], features['y']) # features['x'] * features['y']
    features['dist_xy'] =  tf.sqrt(tf.squared_difference(features['x'],features['y']))
    
    return features

b. Data pipeline input function





In [5]:



    


def tfrecods_input_fn(files_name_pattern, mode=tf.estimator.ModeKeys.EVAL, 
                 num_epochs=None, 
                 batch_size=200):
    
    shuffle = True if mode == tf.estimator.ModeKeys.TRAIN else False
    
    print("")
    print("* data input_fn:")
    print("================")
    print("Input file(s): {}".format(files_name_pattern))
    print("Batch size: {}".format(batch_size))
    print("Epoch Count: {}".format(num_epochs))
    print("Mode: {}".format(mode))
    print("Shuffle: {}".format(shuffle))
    print("================")
    print("")

    file_names = tf.matching_files(files_name_pattern)
    dataset = data.TFRecordDataset(filenames=file_names)

    if shuffle:
        dataset = dataset.shuffle(buffer_size=2 * batch_size + 1)
    
    dataset = dataset.batch(batch_size)
    dataset = dataset.map(lambda tf_example: parse_tf_example(tf_example))
    
    if PROCESS_FEATURES:
        dataset = dataset.map(lambda features, target: (process_features(features), target))
        
    dataset = dataset.repeat(num_epochs)
    iterator = dataset.make_one_shot_iterator()
    
    features, target = iterator.get_next()
    return features, target



    











In [6]:



    


features, target = tfrecods_input_fn(files_name_pattern="")
print("Feature read from TFRecords: {}".format(list(features.keys())))
print("Target read from TFRecords: {}".format(target))



    


















    






* data input_fn:
================
Input file(s): 
Batch size: 200
Epoch Count: None
Mode: eval
Shuffle: False
================

Feature read from TFRecords: ['alpha', 'beta', 'x', 'y', 'x_2', 'y_2', 'xy', 'dist_xy']
Target read from TFRecords: Tensor("IteratorGetNext:8", shape=(?,), dtype=string)

3. Define Feature Columns

The input numeric columns are assumed to be normalized (or have the same scale). Otherise, a normlizer_fn, along with the normlisation params (mean, stdv) should be passed to tf.feature_column.numeric_column() constructor





In [7]:



    


def extend_feature_columns(feature_columns, hparams):
    
    num_buckets = hparams.num_buckets
    embedding_size = hparams.embedding_size

    buckets = np.linspace(-3, 3, num_buckets).tolist()

    alpha_X_beta = tf.feature_column.crossed_column(
            [feature_columns['alpha'], feature_columns['beta']], 4)

    x_bucketized = tf.feature_column.bucketized_column(
            feature_columns['x'], boundaries=buckets)

    y_bucketized = tf.feature_column.bucketized_column(
            feature_columns['y'], boundaries=buckets)

    x_bucketized_X_y_bucketized = tf.feature_column.crossed_column(
           [x_bucketized, y_bucketized], num_buckets**2)

    x_bucketized_X_y_bucketized_embedded = tf.feature_column.embedding_column(
            x_bucketized_X_y_bucketized, dimension=embedding_size)


    feature_columns['alpha_X_beta'] = alpha_X_beta
    feature_columns['x_bucketized_X_y_bucketized'] = x_bucketized_X_y_bucketized
    feature_columns['x_bucketized_X_y_bucketized_embedded'] = x_bucketized_X_y_bucketized_embedded
    
    return feature_columns
    

def get_feature_columns(hparams):
    
    CONSTRUCTED_NUMERIC_FEATURES_NAMES = ['x_2', 'y_2', 'xy', 'dist_xy']
    all_numeric_feature_names = NUMERIC_FEATURE_NAMES.copy() 
    
    if PROCESS_FEATURES:
        all_numeric_feature_names += CONSTRUCTED_NUMERIC_FEATURES_NAMES

    numeric_columns = {feature_name: tf.feature_column.numeric_column(feature_name)
                       for feature_name in all_numeric_feature_names}

    categorical_column_with_vocabulary = \
        {item[0]: tf.feature_column.categorical_column_with_vocabulary_list(item[0], item[1])
         for item in CATEGORICAL_FEATURE_NAMES_WITH_VOCABULARY.items()}
        
    feature_columns = {}

    if numeric_columns is not None:
        feature_columns.update(numeric_columns)

    if categorical_column_with_vocabulary is not None:
        feature_columns.update(categorical_column_with_vocabulary)
    
    if EXTEND_FEATURE_COLUMNS:
        feature_columns = extend_feature_columns(feature_columns, hparams)
        
    return feature_columns

feature_columns = get_feature_columns(tf.contrib.training.HParams(num_buckets=5,embedding_size=3))
print("Feature Columns: {}".format(feature_columns))



    


















    






Feature Columns: {'x': _NumericColumn(key='x', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'y': _NumericColumn(key='y', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'x_2': _NumericColumn(key='x_2', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'y_2': _NumericColumn(key='y_2', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'xy': _NumericColumn(key='xy', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'dist_xy': _NumericColumn(key='dist_xy', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), 'alpha': _VocabularyListCategoricalColumn(key='alpha', vocabulary_list=('ax01', 'ax02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), 'beta': _VocabularyListCategoricalColumn(key='beta', vocabulary_list=('bx01', 'bx02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), 'alpha_X_beta': _CrossedColumn(keys=(_VocabularyListCategoricalColumn(key='alpha', vocabulary_list=('ax01', 'ax02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), _VocabularyListCategoricalColumn(key='beta', vocabulary_list=('bx01', 'bx02'), dtype=tf.string, default_value=-1, num_oov_buckets=0)), hash_bucket_size=4, hash_key=None), 'x_bucketized_X_y_bucketized': _CrossedColumn(keys=(_BucketizedColumn(source_column=_NumericColumn(key='x', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(-3.0, -1.5, 0.0, 1.5, 3.0)), _BucketizedColumn(source_column=_NumericColumn(key='y', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(-3.0, -1.5, 0.0, 1.5, 3.0))), hash_bucket_size=25, hash_key=None), 'x_bucketized_X_y_bucketized_embedded': _EmbeddingColumn(categorical_column=_CrossedColumn(keys=(_BucketizedColumn(source_column=_NumericColumn(key='x', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(-3.0, -1.5, 0.0, 1.5, 3.0)), _BucketizedColumn(source_column=_NumericColumn(key='y', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(-3.0, -1.5, 0.0, 1.5, 3.0))), hash_bucket_size=25, hash_key=None), dimension=3, combiner='mean', initializer=<tensorflow.python.ops.init_ops.TruncatedNormal object at 0x11759f2e8>, ckpt_to_load_from=None, tensor_name_in_ckpt=None, max_norm=None, trainable=True)}

4. Define an Estimator Creation Function

a. Get wide and deep feature columns

  • dense columns = numeric columns + embedding columns
  • categorical columns = vocabolary list columns + bucketized columns
  • sparse columns = hashed categorical columns + crossed columns
  • categorical columns => indicator columns
  • deep columns = dense columns + indicator columns
  • wide columns = categorical columns + sparse columns




In [8]:



    


def get_wide_deep_columns():
    
    feature_columns = list(get_feature_columns(hparams).values())
    
    dense_columns = list(
        filter(lambda column: isinstance(column, feature_column._NumericColumn) |
                              isinstance(column, feature_column._EmbeddingColumn),
               feature_columns
        )
    )

    categorical_columns = list(
        filter(lambda column: isinstance(column, feature_column._VocabularyListCategoricalColumn) |
                              isinstance(column, feature_column._BucketizedColumn),
                   feature_columns)
    )
    
    sparse_columns = list(
        filter(lambda column: isinstance(column,feature_column._HashedCategoricalColumn) |
                              isinstance(column, feature_column._CrossedColumn),
               feature_columns)
    )

    indicator_columns = list(
            map(lambda column: tf.feature_column.indicator_column(column),
                categorical_columns)
    )
    
    deep_feature_columns = dense_columns + indicator_columns
    wide_feature_columns = categorical_columns + sparse_columns
    
    return wide_feature_columns, deep_feature_columns

b. Define the estimator





In [9]:



    


def create_estimator(run_config, hparams, print_desc=False):
    
    wide_feature_columns, deep_feature_columns = get_wide_deep_columns()
    
    estimator = tf.estimator.DNNLinearCombinedClassifier(
        
        n_classes= len(TARGET_LABELS),
        label_vocabulary=TARGET_LABELS,
        
        dnn_feature_columns = deep_feature_columns,
        linear_feature_columns = wide_feature_columns,
        
        dnn_hidden_units= hparams.hidden_units,
        
        dnn_optimizer= tf.train.AdamOptimizer(),
        
        dnn_activation_fn= tf.nn.elu,
        dnn_dropout= hparams.dropout_prob,
        
        config= run_config
    )
    
    if print_desc:
        print("")
        print("*Estimator Type:")
        print("================")
        print(type(estimator))
        print("")
        print("*deep columns:")
        print("==============")
        print(deep_feature_columns)
        print("")
        print("wide columns:")
        print("=============")
        print(wide_feature_columns)
        print("")
    
    return estimator

5. Run Experiment

a. Set HParam and RunConfig





In [10]:



    


TRAIN_SIZE = 12000
NUM_EPOCHS = 1000
BATCH_SIZE = 500
NUM_EVAL = 10
TOTAL_STEPS = (TRAIN_SIZE/BATCH_SIZE)*NUM_EPOCHS
CHECKPOINT_STEPS = int((TRAIN_SIZE/BATCH_SIZE) * (NUM_EPOCHS/NUM_EVAL))

hparams  = tf.contrib.training.HParams(
    num_epochs = NUM_EPOCHS,
    batch_size = BATCH_SIZE,
    hidden_units=[16, 12, 8],
    num_buckets = 6,
    embedding_size = 3,
    max_steps = TOTAL_STEPS,
    dropout_prob = 0.001)

model_dir = 'trained_models/{}'.format(MODEL_NAME)

run_config = tf.contrib.learn.RunConfig(
    save_checkpoints_steps=CHECKPOINT_STEPS,
    tf_random_seed=19830610,
    model_dir=model_dir
)

print(hparams)
print("Model Directory:", run_config.model_dir)
print("")
print("Dataset Size:", TRAIN_SIZE)
print("Batch Size:", BATCH_SIZE)
print("Steps per Epoch:",TRAIN_SIZE/BATCH_SIZE)
print("Total Steps:", TOTAL_STEPS)
print("Required Evaluation Steps:", NUM_EVAL) 
print("That is 1 evaluation step after each",NUM_EPOCHS/NUM_EVAL," epochs")
print("Save Checkpoint After",CHECKPOINT_STEPS,"steps")



    


















    






[('batch_size', 500), ('dropout_prob', 0.001), ('embedding_size', 3), ('hidden_units', [16, 12, 8]), ('max_steps', 24000.0), ('num_buckets', 6), ('num_epochs', 1000)]
Model Directory: trained_models/class-model-01

Dataset Size: 12000
Batch Size: 500
Steps per Epoch: 24.0
Total Steps: 24000.0
Required Evaluation Steps: 10
That is 1 evaluation step after each 100.0  epochs
Save Checkpoint After 2400 steps

b. Define serving function





In [11]:



    


def csv_serving_input_fn():
    
    SERVING_HEADER = ['x','y','alpha','beta']
    SERVING_HEADER_DEFAULTS = [[0.0], [0.0], ['NA'], ['NA']]

    rows_string_tensor = tf.placeholder(dtype=tf.string,
                                         shape=[None],
                                         name='csv_rows')
    
    receiver_tensor = {'csv_rows': rows_string_tensor}

    row_columns = tf.expand_dims(rows_string_tensor, -1)
    columns = tf.decode_csv(row_columns, record_defaults=SERVING_HEADER_DEFAULTS)
    features = dict(zip(SERVING_HEADER, columns))

    return tf.estimator.export.ServingInputReceiver(
        process_features(features), receiver_tensor)

c. Define TrainSpec and EvaluSpec





In [12]:



    


train_spec = tf.estimator.TrainSpec(
    input_fn = lambda: tfrecods_input_fn(
        TRAIN_DATA_FILES_PATTERN,
        mode = tf.estimator.ModeKeys.TRAIN,
        num_epochs=hparams.num_epochs,
        batch_size=hparams.batch_size
    ),
    max_steps=hparams.max_steps,
    hooks=None
)

eval_spec = tf.estimator.EvalSpec(
    input_fn = lambda: tfrecods_input_fn(
        VALID_DATA_FILES_PATTERN,
        mode=tf.estimator.ModeKeys.EVAL,
        num_epochs=1,
        batch_size=hparams.batch_size
    ),
    exporters=[tf.estimator.LatestExporter(
        name="predict", # the name of the folder in which the model will be exported to under export
        serving_input_receiver_fn=csv_serving_input_fn,
        exports_to_keep=1,
        as_text=True)],
    steps=None,
    hooks=None
)

d. Run the experiment via train_and_evaluate





In [13]:



    


if not RESUME_TRAINING:
    print("Removing previous artifacts...")
    shutil.rmtree(model_dir, ignore_errors=True)
else:
    print("Resuming training...") 

    
tf.logging.set_verbosity(tf.logging.INFO)

time_start = datetime.utcnow() 
print("Experiment started at {}".format(time_start.strftime("%H:%M:%S")))
print(".......................................") 

estimator = create_estimator(run_config, hparams, True)

tf.estimator.train_and_evaluate(
    estimator=estimator,
    train_spec=train_spec, 
    eval_spec=eval_spec
)

time_end = datetime.utcnow() 
print(".......................................")
print("Experiment finished at {}".format(time_end.strftime("%H:%M:%S")))
print("")
time_elapsed = time_end - time_start
print("Experiment elapsed time: {} seconds".format(time_elapsed.total_seconds()))



    


















    






Removing previous artifacts...
Experiment started at 18:33:38
.......................................
INFO:tensorflow:Using config: {'_task_type': None, '_task_id': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x117584b70>, '_master': '', '_num_ps_replicas': 0, '_num_worker_replicas': 0, '_environment': 'local', '_is_chief': True, '_evaluation_master': '', '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1
}
, '_tf_random_seed': 19830610, '_save_summary_steps': 100, '_save_checkpoints_secs': None, '_log_step_count_steps': 100, '_session_config': None, '_save_checkpoints_steps': 2400, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_model_dir': 'trained_models/class-model-01'}

*Estimator Type:
================
<class 'tensorflow.python.estimator.canned.dnn_linear_combined.DNNLinearCombinedClassifier'>

*deep columns:
==============
[_NumericColumn(key='x', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), _NumericColumn(key='y', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), _NumericColumn(key='x_2', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), _NumericColumn(key='y_2', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), _NumericColumn(key='xy', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), _NumericColumn(key='dist_xy', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), _EmbeddingColumn(categorical_column=_CrossedColumn(keys=(_BucketizedColumn(source_column=_NumericColumn(key='x', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(-3.0, -1.8, -0.6000000000000001, 0.5999999999999996, 1.7999999999999998, 3.0)), _BucketizedColumn(source_column=_NumericColumn(key='y', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(-3.0, -1.8, -0.6000000000000001, 0.5999999999999996, 1.7999999999999998, 3.0))), hash_bucket_size=36, hash_key=None), dimension=3, combiner='mean', initializer=<tensorflow.python.ops.init_ops.TruncatedNormal object at 0x119ad9a90>, ckpt_to_load_from=None, tensor_name_in_ckpt=None, max_norm=None, trainable=True), _IndicatorColumn(categorical_column=_VocabularyListCategoricalColumn(key='alpha', vocabulary_list=('ax01', 'ax02'), dtype=tf.string, default_value=-1, num_oov_buckets=0)), _IndicatorColumn(categorical_column=_VocabularyListCategoricalColumn(key='beta', vocabulary_list=('bx01', 'bx02'), dtype=tf.string, default_value=-1, num_oov_buckets=0))]

wide columns:
=============
[_VocabularyListCategoricalColumn(key='alpha', vocabulary_list=('ax01', 'ax02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), _VocabularyListCategoricalColumn(key='beta', vocabulary_list=('bx01', 'bx02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), _CrossedColumn(keys=(_VocabularyListCategoricalColumn(key='alpha', vocabulary_list=('ax01', 'ax02'), dtype=tf.string, default_value=-1, num_oov_buckets=0), _VocabularyListCategoricalColumn(key='beta', vocabulary_list=('bx01', 'bx02'), dtype=tf.string, default_value=-1, num_oov_buckets=0)), hash_bucket_size=4, hash_key=None), _CrossedColumn(keys=(_BucketizedColumn(source_column=_NumericColumn(key='x', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(-3.0, -1.8, -0.6000000000000001, 0.5999999999999996, 1.7999999999999998, 3.0)), _BucketizedColumn(source_column=_NumericColumn(key='y', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(-3.0, -1.8, -0.6000000000000001, 0.5999999999999996, 1.7999999999999998, 3.0))), hash_bucket_size=36, hash_key=None)]

INFO:tensorflow:Running training and evaluation locally (non-distributed).
INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after 600 secs (eval_spec.throttle_secs) or training is finished.

* data input_fn:
================
Input file(s): data/train-*.tfrecords
Batch size: 500
Epoch Count: 1000
Mode: train
Shuffle: True
================

INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Saving checkpoints for 1 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:loss = 367.798, step = 1
INFO:tensorflow:global_step/sec: 131.567
INFO:tensorflow:loss = 300.208, step = 101 (0.762 sec)
INFO:tensorflow:global_step/sec: 217.539
INFO:tensorflow:loss = 211.619, step = 201 (0.460 sec)
INFO:tensorflow:global_step/sec: 203.007
INFO:tensorflow:loss = 162.349, step = 301 (0.494 sec)
INFO:tensorflow:global_step/sec: 168.143
INFO:tensorflow:loss = 124.463, step = 401 (0.593 sec)
INFO:tensorflow:global_step/sec: 155.649
INFO:tensorflow:loss = 122.562, step = 501 (0.643 sec)
INFO:tensorflow:global_step/sec: 189.159
INFO:tensorflow:loss = 83.5131, step = 601 (0.528 sec)
INFO:tensorflow:global_step/sec: 204.765
INFO:tensorflow:loss = 84.2995, step = 701 (0.489 sec)
INFO:tensorflow:global_step/sec: 228.929
INFO:tensorflow:loss = 65.9374, step = 801 (0.436 sec)
INFO:tensorflow:global_step/sec: 206.429
INFO:tensorflow:loss = 74.7193, step = 901 (0.486 sec)
INFO:tensorflow:global_step/sec: 158.396
INFO:tensorflow:loss = 80.7219, step = 1001 (0.631 sec)
INFO:tensorflow:global_step/sec: 162.983
INFO:tensorflow:loss = 66.1648, step = 1101 (0.613 sec)
INFO:tensorflow:global_step/sec: 217.549
INFO:tensorflow:loss = 54.1094, step = 1201 (0.462 sec)
INFO:tensorflow:global_step/sec: 227.447
INFO:tensorflow:loss = 53.8291, step = 1301 (0.437 sec)
INFO:tensorflow:global_step/sec: 224.884
INFO:tensorflow:loss = 59.0117, step = 1401 (0.446 sec)
INFO:tensorflow:global_step/sec: 174.015
INFO:tensorflow:loss = 67.3785, step = 1501 (0.579 sec)
INFO:tensorflow:global_step/sec: 217.273
INFO:tensorflow:loss = 68.933, step = 1601 (0.456 sec)
INFO:tensorflow:global_step/sec: 207.45
INFO:tensorflow:loss = 64.3857, step = 1701 (0.480 sec)
INFO:tensorflow:global_step/sec: 264.146
INFO:tensorflow:loss = 51.0939, step = 1801 (0.379 sec)
INFO:tensorflow:global_step/sec: 267.469
INFO:tensorflow:loss = 44.0319, step = 1901 (0.374 sec)
INFO:tensorflow:global_step/sec: 274.885
INFO:tensorflow:loss = 55.9877, step = 2001 (0.365 sec)
INFO:tensorflow:global_step/sec: 256.865
INFO:tensorflow:loss = 42.752, step = 2101 (0.388 sec)
INFO:tensorflow:global_step/sec: 267.256
INFO:tensorflow:loss = 59.587, step = 2201 (0.375 sec)
INFO:tensorflow:global_step/sec: 273.152
INFO:tensorflow:loss = 59.8185, step = 2301 (0.366 sec)
INFO:tensorflow:Saving checkpoints for 2401 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 60.4536
INFO:tensorflow:loss = 43.2125, step = 2401 (1.654 sec)
INFO:tensorflow:global_step/sec: 266.203
INFO:tensorflow:loss = 51.4669, step = 2501 (0.376 sec)
INFO:tensorflow:global_step/sec: 267.544
INFO:tensorflow:loss = 62.8036, step = 2601 (0.374 sec)
INFO:tensorflow:global_step/sec: 271.808
INFO:tensorflow:loss = 60.0741, step = 2701 (0.369 sec)
INFO:tensorflow:global_step/sec: 268.904
INFO:tensorflow:loss = 89.6466, step = 2801 (0.371 sec)
INFO:tensorflow:global_step/sec: 268.363
INFO:tensorflow:loss = 55.1014, step = 2901 (0.372 sec)
INFO:tensorflow:global_step/sec: 269.118
INFO:tensorflow:loss = 56.6122, step = 3001 (0.372 sec)
INFO:tensorflow:global_step/sec: 270.157
INFO:tensorflow:loss = 57.6482, step = 3101 (0.370 sec)
INFO:tensorflow:global_step/sec: 273.17
INFO:tensorflow:loss = 54.1119, step = 3201 (0.366 sec)
INFO:tensorflow:global_step/sec: 271.78
INFO:tensorflow:loss = 61.9126, step = 3301 (0.368 sec)
INFO:tensorflow:global_step/sec: 271.568
INFO:tensorflow:loss = 53.3086, step = 3401 (0.368 sec)
INFO:tensorflow:global_step/sec: 268.484
INFO:tensorflow:loss = 74.519, step = 3501 (0.373 sec)
INFO:tensorflow:global_step/sec: 275.541
INFO:tensorflow:loss = 53.2911, step = 3601 (0.364 sec)
INFO:tensorflow:global_step/sec: 269.438
INFO:tensorflow:loss = 56.8043, step = 3701 (0.371 sec)
INFO:tensorflow:global_step/sec: 257.584
INFO:tensorflow:loss = 50.6941, step = 3801 (0.388 sec)
INFO:tensorflow:global_step/sec: 264.237
INFO:tensorflow:loss = 63.9508, step = 3901 (0.378 sec)
INFO:tensorflow:global_step/sec: 265.679
INFO:tensorflow:loss = 68.9809, step = 4001 (0.376 sec)
INFO:tensorflow:global_step/sec: 264.127
INFO:tensorflow:loss = 58.1893, step = 4101 (0.379 sec)
INFO:tensorflow:global_step/sec: 263.797
INFO:tensorflow:loss = 47.9043, step = 4201 (0.379 sec)
INFO:tensorflow:global_step/sec: 264.367
INFO:tensorflow:loss = 54.2471, step = 4301 (0.378 sec)
INFO:tensorflow:global_step/sec: 264.074
INFO:tensorflow:loss = 40.7804, step = 4401 (0.379 sec)
INFO:tensorflow:global_step/sec: 264.185
INFO:tensorflow:loss = 48.1839, step = 4501 (0.379 sec)
INFO:tensorflow:global_step/sec: 264.204
INFO:tensorflow:loss = 73.7359, step = 4601 (0.378 sec)
INFO:tensorflow:global_step/sec: 238.013
INFO:tensorflow:loss = 54.733, step = 4701 (0.420 sec)
INFO:tensorflow:Saving checkpoints for 4801 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 51.8358
INFO:tensorflow:loss = 53.9156, step = 4801 (1.932 sec)
INFO:tensorflow:global_step/sec: 224.411
INFO:tensorflow:loss = 55.8483, step = 4901 (0.445 sec)
INFO:tensorflow:global_step/sec: 233.492
INFO:tensorflow:loss = 40.7985, step = 5001 (0.426 sec)
INFO:tensorflow:global_step/sec: 230.706
INFO:tensorflow:loss = 54.558, step = 5101 (0.434 sec)
INFO:tensorflow:global_step/sec: 236.828
INFO:tensorflow:loss = 60.9781, step = 5201 (0.422 sec)
INFO:tensorflow:global_step/sec: 234.714
INFO:tensorflow:loss = 67.3702, step = 5301 (0.426 sec)
INFO:tensorflow:global_step/sec: 237.645
INFO:tensorflow:loss = 56.3291, step = 5401 (0.421 sec)
INFO:tensorflow:global_step/sec: 235.311
INFO:tensorflow:loss = 48.3456, step = 5501 (0.425 sec)
INFO:tensorflow:global_step/sec: 235.108
INFO:tensorflow:loss = 49.5535, step = 5601 (0.426 sec)
INFO:tensorflow:global_step/sec: 235.963
INFO:tensorflow:loss = 59.5464, step = 5701 (0.424 sec)
INFO:tensorflow:global_step/sec: 240.435
INFO:tensorflow:loss = 44.8572, step = 5801 (0.416 sec)
INFO:tensorflow:global_step/sec: 235.146
INFO:tensorflow:loss = 53.2054, step = 5901 (0.425 sec)
INFO:tensorflow:global_step/sec: 239.586
INFO:tensorflow:loss = 55.1002, step = 6001 (0.417 sec)
INFO:tensorflow:global_step/sec: 235.897
INFO:tensorflow:loss = 51.0697, step = 6101 (0.424 sec)
INFO:tensorflow:global_step/sec: 240.418
INFO:tensorflow:loss = 48.0757, step = 6201 (0.416 sec)
INFO:tensorflow:global_step/sec: 228.353
INFO:tensorflow:loss = 60.4919, step = 6301 (0.438 sec)
INFO:tensorflow:global_step/sec: 235.42
INFO:tensorflow:loss = 64.6721, step = 6401 (0.425 sec)
INFO:tensorflow:global_step/sec: 240.569
INFO:tensorflow:loss = 40.0768, step = 6501 (0.416 sec)
INFO:tensorflow:global_step/sec: 241.492
INFO:tensorflow:loss = 45.7592, step = 6601 (0.416 sec)
INFO:tensorflow:global_step/sec: 234.818
INFO:tensorflow:loss = 60.6002, step = 6701 (0.424 sec)
INFO:tensorflow:global_step/sec: 231.46
INFO:tensorflow:loss = 55.0018, step = 6801 (0.432 sec)
INFO:tensorflow:global_step/sec: 234.027
INFO:tensorflow:loss = 56.4116, step = 6901 (0.427 sec)
INFO:tensorflow:global_step/sec: 232.723
INFO:tensorflow:loss = 50.9101, step = 7001 (0.430 sec)
INFO:tensorflow:global_step/sec: 234.81
INFO:tensorflow:loss = 56.9551, step = 7101 (0.425 sec)
INFO:tensorflow:Saving checkpoints for 7201 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 56.8195
INFO:tensorflow:loss = 52.6766, step = 7201 (1.762 sec)
INFO:tensorflow:global_step/sec: 231.019
INFO:tensorflow:loss = 59.5233, step = 7301 (0.431 sec)
INFO:tensorflow:global_step/sec: 231.699
INFO:tensorflow:loss = 54.5579, step = 7401 (0.431 sec)
INFO:tensorflow:global_step/sec: 233.641
INFO:tensorflow:loss = 47.1151, step = 7501 (0.428 sec)
INFO:tensorflow:global_step/sec: 235.591
INFO:tensorflow:loss = 59.0914, step = 7601 (0.424 sec)
INFO:tensorflow:global_step/sec: 231.872
INFO:tensorflow:loss = 44.9465, step = 7701 (0.432 sec)
INFO:tensorflow:global_step/sec: 235.373
INFO:tensorflow:loss = 49.3429, step = 7801 (0.424 sec)
INFO:tensorflow:global_step/sec: 234.139
INFO:tensorflow:loss = 50.571, step = 7901 (0.429 sec)
INFO:tensorflow:global_step/sec: 232.392
INFO:tensorflow:loss = 35.5534, step = 8001 (0.429 sec)
INFO:tensorflow:global_step/sec: 234.263
INFO:tensorflow:loss = 47.7327, step = 8101 (0.427 sec)
INFO:tensorflow:global_step/sec: 249.173
INFO:tensorflow:loss = 55.5154, step = 8201 (0.401 sec)
INFO:tensorflow:global_step/sec: 259.375
INFO:tensorflow:loss = 45.991, step = 8301 (0.385 sec)
INFO:tensorflow:global_step/sec: 257.509
INFO:tensorflow:loss = 47.9003, step = 8401 (0.388 sec)
INFO:tensorflow:global_step/sec: 261.519
INFO:tensorflow:loss = 56.759, step = 8501 (0.383 sec)
INFO:tensorflow:global_step/sec: 257.747
INFO:tensorflow:loss = 68.8786, step = 8601 (0.389 sec)
INFO:tensorflow:global_step/sec: 260.364
INFO:tensorflow:loss = 62.8279, step = 8701 (0.383 sec)
INFO:tensorflow:global_step/sec: 258.259
INFO:tensorflow:loss = 66.1603, step = 8801 (0.388 sec)
INFO:tensorflow:global_step/sec: 259.311
INFO:tensorflow:loss = 53.6692, step = 8901 (0.385 sec)
INFO:tensorflow:global_step/sec: 260.693
INFO:tensorflow:loss = 56.0062, step = 9001 (0.384 sec)
INFO:tensorflow:global_step/sec: 259.597
INFO:tensorflow:loss = 60.9664, step = 9101 (0.385 sec)
INFO:tensorflow:global_step/sec: 258.059
INFO:tensorflow:loss = 33.8594, step = 9201 (0.388 sec)
INFO:tensorflow:global_step/sec: 261.769
INFO:tensorflow:loss = 40.6498, step = 9301 (0.382 sec)
INFO:tensorflow:global_step/sec: 259.707
INFO:tensorflow:loss = 59.9937, step = 9401 (0.385 sec)
INFO:tensorflow:global_step/sec: 259.973
INFO:tensorflow:loss = 64.3405, step = 9501 (0.384 sec)
INFO:tensorflow:Saving checkpoints for 9601 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 60.1905
INFO:tensorflow:loss = 47.7602, step = 9601 (1.661 sec)
INFO:tensorflow:global_step/sec: 256.376
INFO:tensorflow:loss = 63.218, step = 9701 (0.390 sec)
INFO:tensorflow:global_step/sec: 263.816
INFO:tensorflow:loss = 47.6366, step = 9801 (0.380 sec)
INFO:tensorflow:global_step/sec: 258.752
INFO:tensorflow:loss = 54.2859, step = 9901 (0.386 sec)
INFO:tensorflow:global_step/sec: 255.746
INFO:tensorflow:loss = 64.6654, step = 10001 (0.391 sec)
INFO:tensorflow:global_step/sec: 260.819
INFO:tensorflow:loss = 63.5053, step = 10101 (0.383 sec)
INFO:tensorflow:global_step/sec: 258.927
INFO:tensorflow:loss = 59.1471, step = 10201 (0.386 sec)
INFO:tensorflow:global_step/sec: 250.153
INFO:tensorflow:loss = 60.5707, step = 10301 (0.400 sec)
INFO:tensorflow:global_step/sec: 250.005
INFO:tensorflow:loss = 60.2314, step = 10401 (0.400 sec)
INFO:tensorflow:global_step/sec: 253.572
INFO:tensorflow:loss = 35.174, step = 10501 (0.395 sec)
INFO:tensorflow:global_step/sec: 259.843
INFO:tensorflow:loss = 45.5049, step = 10601 (0.384 sec)
INFO:tensorflow:global_step/sec: 263.736
INFO:tensorflow:loss = 61.0202, step = 10701 (0.380 sec)
INFO:tensorflow:global_step/sec: 259.38
INFO:tensorflow:loss = 41.4451, step = 10801 (0.385 sec)
INFO:tensorflow:global_step/sec: 262.452
INFO:tensorflow:loss = 65.7566, step = 10901 (0.381 sec)
INFO:tensorflow:global_step/sec: 260.182
INFO:tensorflow:loss = 54.438, step = 11001 (0.385 sec)
INFO:tensorflow:global_step/sec: 257.292
INFO:tensorflow:loss = 47.2766, step = 11101 (0.389 sec)
INFO:tensorflow:global_step/sec: 260.139
INFO:tensorflow:loss = 62.1429, step = 11201 (0.385 sec)
INFO:tensorflow:global_step/sec: 259.334
INFO:tensorflow:loss = 47.4337, step = 11301 (0.385 sec)
INFO:tensorflow:global_step/sec: 258.361
INFO:tensorflow:loss = 45.8335, step = 11401 (0.388 sec)
INFO:tensorflow:global_step/sec: 258.29
INFO:tensorflow:loss = 41.1917, step = 11501 (0.386 sec)
INFO:tensorflow:global_step/sec: 258.232
INFO:tensorflow:loss = 45.1506, step = 11601 (0.387 sec)
INFO:tensorflow:global_step/sec: 258.616
INFO:tensorflow:loss = 57.6406, step = 11701 (0.387 sec)
INFO:tensorflow:global_step/sec: 257.735
INFO:tensorflow:loss = 62.2266, step = 11801 (0.388 sec)
INFO:tensorflow:global_step/sec: 259.063
INFO:tensorflow:loss = 46.3194, step = 11901 (0.386 sec)
INFO:tensorflow:Saving checkpoints for 12001 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 60.2744
INFO:tensorflow:loss = 40.8159, step = 12001 (1.659 sec)
INFO:tensorflow:global_step/sec: 254.821
INFO:tensorflow:loss = 37.4166, step = 12101 (0.393 sec)
INFO:tensorflow:global_step/sec: 257.767
INFO:tensorflow:loss = 50.3235, step = 12201 (0.388 sec)
INFO:tensorflow:global_step/sec: 261.623
INFO:tensorflow:loss = 52.2078, step = 12301 (0.382 sec)
INFO:tensorflow:global_step/sec: 257.136
INFO:tensorflow:loss = 58.3317, step = 12401 (0.389 sec)
INFO:tensorflow:global_step/sec: 256.514
INFO:tensorflow:loss = 56.406, step = 12501 (0.390 sec)
INFO:tensorflow:global_step/sec: 258.975
INFO:tensorflow:loss = 62.1642, step = 12601 (0.387 sec)
INFO:tensorflow:global_step/sec: 256.882
INFO:tensorflow:loss = 44.5875, step = 12701 (0.389 sec)
INFO:tensorflow:global_step/sec: 261.261
INFO:tensorflow:loss = 44.3237, step = 12801 (0.383 sec)
INFO:tensorflow:global_step/sec: 259.36
INFO:tensorflow:loss = 43.1127, step = 12901 (0.385 sec)
INFO:tensorflow:global_step/sec: 262.536
INFO:tensorflow:loss = 73.5175, step = 13001 (0.381 sec)
INFO:tensorflow:global_step/sec: 259.085
INFO:tensorflow:loss = 55.8118, step = 13101 (0.386 sec)
INFO:tensorflow:global_step/sec: 257.946
INFO:tensorflow:loss = 50.3784, step = 13201 (0.388 sec)
INFO:tensorflow:global_step/sec: 258.707
INFO:tensorflow:loss = 47.194, step = 13301 (0.387 sec)
INFO:tensorflow:global_step/sec: 259.348
INFO:tensorflow:loss = 43.2756, step = 13401 (0.386 sec)
INFO:tensorflow:global_step/sec: 259.919
INFO:tensorflow:loss = 55.137, step = 13501 (0.384 sec)
INFO:tensorflow:global_step/sec: 259.154
INFO:tensorflow:loss = 51.9417, step = 13601 (0.386 sec)
INFO:tensorflow:global_step/sec: 259.498
INFO:tensorflow:loss = 65.6125, step = 13701 (0.385 sec)
INFO:tensorflow:global_step/sec: 259.342
INFO:tensorflow:loss = 59.7105, step = 13801 (0.386 sec)
INFO:tensorflow:global_step/sec: 261.359
INFO:tensorflow:loss = 59.71, step = 13901 (0.382 sec)
INFO:tensorflow:global_step/sec: 259.925
INFO:tensorflow:loss = 41.7235, step = 14001 (0.385 sec)
INFO:tensorflow:global_step/sec: 263.586
INFO:tensorflow:loss = 62.3289, step = 14101 (0.380 sec)
INFO:tensorflow:global_step/sec: 260.605
INFO:tensorflow:loss = 53.7547, step = 14201 (0.383 sec)
INFO:tensorflow:global_step/sec: 257.689
INFO:tensorflow:loss = 54.4652, step = 14301 (0.388 sec)
INFO:tensorflow:Saving checkpoints for 14401 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 56.4173
INFO:tensorflow:loss = 51.5741, step = 14401 (1.774 sec)
INFO:tensorflow:global_step/sec: 223.132
INFO:tensorflow:loss = 54.753, step = 14501 (0.448 sec)
INFO:tensorflow:global_step/sec: 235.529
INFO:tensorflow:loss = 61.0252, step = 14601 (0.425 sec)
INFO:tensorflow:global_step/sec: 230.584
INFO:tensorflow:loss = 64.008, step = 14701 (0.433 sec)
INFO:tensorflow:global_step/sec: 233.992
INFO:tensorflow:loss = 49.3189, step = 14801 (0.428 sec)
INFO:tensorflow:global_step/sec: 223.536
INFO:tensorflow:loss = 58.4319, step = 14901 (0.447 sec)
INFO:tensorflow:global_step/sec: 228.351
INFO:tensorflow:loss = 56.4073, step = 15001 (0.438 sec)
INFO:tensorflow:global_step/sec: 234.848
INFO:tensorflow:loss = 59.6387, step = 15101 (0.426 sec)
INFO:tensorflow:global_step/sec: 235.891
INFO:tensorflow:loss = 67.3168, step = 15201 (0.424 sec)
INFO:tensorflow:global_step/sec: 234.758
INFO:tensorflow:loss = 53.2103, step = 15301 (0.426 sec)
INFO:tensorflow:global_step/sec: 181.644
INFO:tensorflow:loss = 69.8954, step = 15401 (0.551 sec)
INFO:tensorflow:global_step/sec: 210.363
INFO:tensorflow:loss = 53.9213, step = 15501 (0.475 sec)
INFO:tensorflow:global_step/sec: 235.161
INFO:tensorflow:loss = 49.8122, step = 15601 (0.425 sec)
INFO:tensorflow:global_step/sec: 224.361
INFO:tensorflow:loss = 59.6161, step = 15701 (0.446 sec)
INFO:tensorflow:global_step/sec: 234.406
INFO:tensorflow:loss = 48.2354, step = 15801 (0.426 sec)
INFO:tensorflow:global_step/sec: 233.078
INFO:tensorflow:loss = 41.5582, step = 15901 (0.429 sec)
INFO:tensorflow:global_step/sec: 222.225
INFO:tensorflow:loss = 59.6899, step = 16001 (0.450 sec)
INFO:tensorflow:global_step/sec: 229.377
INFO:tensorflow:loss = 47.8183, step = 16101 (0.436 sec)
INFO:tensorflow:global_step/sec: 234.012
INFO:tensorflow:loss = 48.2806, step = 16201 (0.427 sec)
INFO:tensorflow:global_step/sec: 238.023
INFO:tensorflow:loss = 40.7605, step = 16301 (0.420 sec)
INFO:tensorflow:global_step/sec: 233.975
INFO:tensorflow:loss = 49.5315, step = 16401 (0.428 sec)
INFO:tensorflow:global_step/sec: 232.884
INFO:tensorflow:loss = 53.5297, step = 16501 (0.429 sec)
INFO:tensorflow:global_step/sec: 235.717
INFO:tensorflow:loss = 46.6379, step = 16601 (0.426 sec)
INFO:tensorflow:global_step/sec: 230.535
INFO:tensorflow:loss = 57.2792, step = 16701 (0.433 sec)
INFO:tensorflow:Saving checkpoints for 16801 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 56.9906
INFO:tensorflow:loss = 42.8698, step = 16801 (1.755 sec)
INFO:tensorflow:global_step/sec: 231.039
INFO:tensorflow:loss = 50.3401, step = 16901 (0.433 sec)
INFO:tensorflow:global_step/sec: 233.743
INFO:tensorflow:loss = 44.6719, step = 17001 (0.426 sec)
INFO:tensorflow:global_step/sec: 175.317
INFO:tensorflow:loss = 33.619, step = 17101 (0.572 sec)
INFO:tensorflow:global_step/sec: 222.652
INFO:tensorflow:loss = 58.4876, step = 17201 (0.447 sec)
INFO:tensorflow:global_step/sec: 234.182
INFO:tensorflow:loss = 70.57, step = 17301 (0.428 sec)
INFO:tensorflow:global_step/sec: 231.747
INFO:tensorflow:loss = 50.9063, step = 17401 (0.430 sec)
INFO:tensorflow:global_step/sec: 234.392
INFO:tensorflow:loss = 46.2028, step = 17501 (0.427 sec)
INFO:tensorflow:global_step/sec: 233.075
INFO:tensorflow:loss = 39.7614, step = 17601 (0.429 sec)
INFO:tensorflow:global_step/sec: 231.437
INFO:tensorflow:loss = 48.1493, step = 17701 (0.433 sec)
INFO:tensorflow:global_step/sec: 234.511
INFO:tensorflow:loss = 54.7027, step = 17801 (0.425 sec)
INFO:tensorflow:global_step/sec: 237.679
INFO:tensorflow:loss = 55.7525, step = 17901 (0.421 sec)
INFO:tensorflow:global_step/sec: 231.094
INFO:tensorflow:loss = 48.5197, step = 18001 (0.433 sec)
INFO:tensorflow:global_step/sec: 228.301
INFO:tensorflow:loss = 47.9747, step = 18101 (0.439 sec)
INFO:tensorflow:global_step/sec: 234.437
INFO:tensorflow:loss = 32.2431, step = 18201 (0.426 sec)
INFO:tensorflow:global_step/sec: 237.698
INFO:tensorflow:loss = 49.7439, step = 18301 (0.420 sec)
INFO:tensorflow:global_step/sec: 234.51
INFO:tensorflow:loss = 52.7422, step = 18401 (0.427 sec)
INFO:tensorflow:global_step/sec: 236.19
INFO:tensorflow:loss = 51.0026, step = 18501 (0.423 sec)
INFO:tensorflow:global_step/sec: 236.766
INFO:tensorflow:loss = 51.0279, step = 18601 (0.422 sec)
INFO:tensorflow:global_step/sec: 237.121
INFO:tensorflow:loss = 67.6928, step = 18701 (0.422 sec)
INFO:tensorflow:global_step/sec: 235.737
INFO:tensorflow:loss = 40.5942, step = 18801 (0.424 sec)
INFO:tensorflow:global_step/sec: 231.463
INFO:tensorflow:loss = 76.1572, step = 18901 (0.432 sec)
INFO:tensorflow:global_step/sec: 234.572
INFO:tensorflow:loss = 57.2259, step = 19001 (0.426 sec)
INFO:tensorflow:global_step/sec: 234.314
INFO:tensorflow:loss = 52.327, step = 19101 (0.427 sec)
INFO:tensorflow:Saving checkpoints for 19201 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 52.9564
INFO:tensorflow:loss = 44.078, step = 19201 (1.889 sec)
INFO:tensorflow:global_step/sec: 232.134
INFO:tensorflow:loss = 49.0712, step = 19301 (0.431 sec)
INFO:tensorflow:global_step/sec: 231.229
INFO:tensorflow:loss = 65.2483, step = 19401 (0.431 sec)
INFO:tensorflow:global_step/sec: 230.829
INFO:tensorflow:loss = 41.6333, step = 19501 (0.433 sec)
INFO:tensorflow:global_step/sec: 235.836
INFO:tensorflow:loss = 70.4016, step = 19601 (0.423 sec)
INFO:tensorflow:global_step/sec: 232.652
INFO:tensorflow:loss = 46.5377, step = 19701 (0.430 sec)
INFO:tensorflow:global_step/sec: 231.818
INFO:tensorflow:loss = 46.9032, step = 19801 (0.431 sec)
INFO:tensorflow:global_step/sec: 229.986
INFO:tensorflow:loss = 46.2393, step = 19901 (0.435 sec)
INFO:tensorflow:global_step/sec: 231.823
INFO:tensorflow:loss = 33.4292, step = 20001 (0.432 sec)
INFO:tensorflow:global_step/sec: 222.896
INFO:tensorflow:loss = 51.4318, step = 20101 (0.448 sec)
INFO:tensorflow:global_step/sec: 214.516
INFO:tensorflow:loss = 60.9859, step = 20201 (0.466 sec)
INFO:tensorflow:global_step/sec: 217.665
INFO:tensorflow:loss = 52.5739, step = 20301 (0.460 sec)
INFO:tensorflow:global_step/sec: 226.661
INFO:tensorflow:loss = 50.3749, step = 20401 (0.440 sec)
INFO:tensorflow:global_step/sec: 237.21
INFO:tensorflow:loss = 48.6444, step = 20501 (0.422 sec)
INFO:tensorflow:global_step/sec: 235.135
INFO:tensorflow:loss = 48.185, step = 20601 (0.425 sec)
INFO:tensorflow:global_step/sec: 235.74
INFO:tensorflow:loss = 53.1283, step = 20701 (0.424 sec)
INFO:tensorflow:global_step/sec: 235.015
INFO:tensorflow:loss = 56.8052, step = 20801 (0.426 sec)
INFO:tensorflow:global_step/sec: 234.545
INFO:tensorflow:loss = 51.5109, step = 20901 (0.426 sec)
INFO:tensorflow:global_step/sec: 235.379
INFO:tensorflow:loss = 52.3609, step = 21001 (0.425 sec)
INFO:tensorflow:global_step/sec: 221.871
INFO:tensorflow:loss = 55.9556, step = 21101 (0.451 sec)
INFO:tensorflow:global_step/sec: 231.586
INFO:tensorflow:loss = 51.1583, step = 21201 (0.432 sec)
INFO:tensorflow:global_step/sec: 233.815
INFO:tensorflow:loss = 58.6974, step = 21301 (0.428 sec)
INFO:tensorflow:global_step/sec: 232.088
INFO:tensorflow:loss = 46.9463, step = 21401 (0.431 sec)
INFO:tensorflow:global_step/sec: 228.467
INFO:tensorflow:loss = 61.6762, step = 21501 (0.437 sec)
INFO:tensorflow:Saving checkpoints for 21601 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:global_step/sec: 52.5257
INFO:tensorflow:loss = 49.5881, step = 21601 (1.905 sec)
INFO:tensorflow:global_step/sec: 230.768
INFO:tensorflow:loss = 43.3981, step = 21701 (0.433 sec)
INFO:tensorflow:global_step/sec: 232.808
INFO:tensorflow:loss = 59.7745, step = 21801 (0.429 sec)
INFO:tensorflow:global_step/sec: 232.429
INFO:tensorflow:loss = 47.4286, step = 21901 (0.430 sec)
INFO:tensorflow:global_step/sec: 237.663
INFO:tensorflow:loss = 52.8188, step = 22001 (0.421 sec)
INFO:tensorflow:global_step/sec: 233.688
INFO:tensorflow:loss = 54.1217, step = 22101 (0.429 sec)
INFO:tensorflow:global_step/sec: 235.676
INFO:tensorflow:loss = 46.0651, step = 22201 (0.424 sec)
INFO:tensorflow:global_step/sec: 233.701
INFO:tensorflow:loss = 62.771, step = 22301 (0.428 sec)
INFO:tensorflow:global_step/sec: 238.158
INFO:tensorflow:loss = 46.2049, step = 22401 (0.420 sec)
INFO:tensorflow:global_step/sec: 233.361
INFO:tensorflow:loss = 53.8034, step = 22501 (0.429 sec)
INFO:tensorflow:global_step/sec: 227.542
INFO:tensorflow:loss = 64.6988, step = 22601 (0.439 sec)
INFO:tensorflow:global_step/sec: 227.737
INFO:tensorflow:loss = 74.1382, step = 22701 (0.438 sec)
INFO:tensorflow:global_step/sec: 233.494
INFO:tensorflow:loss = 55.4112, step = 22801 (0.429 sec)
INFO:tensorflow:global_step/sec: 231.499
INFO:tensorflow:loss = 49.725, step = 22901 (0.432 sec)
INFO:tensorflow:global_step/sec: 235.506
INFO:tensorflow:loss = 52.0909, step = 23001 (0.425 sec)
INFO:tensorflow:global_step/sec: 236.832
INFO:tensorflow:loss = 53.1299, step = 23101 (0.422 sec)
INFO:tensorflow:global_step/sec: 233.77
INFO:tensorflow:loss = 41.5742, step = 23201 (0.428 sec)
INFO:tensorflow:global_step/sec: 234.061
INFO:tensorflow:loss = 63.9089, step = 23301 (0.427 sec)
INFO:tensorflow:global_step/sec: 237.172
INFO:tensorflow:loss = 50.7253, step = 23401 (0.421 sec)
INFO:tensorflow:global_step/sec: 235.564
INFO:tensorflow:loss = 49.1982, step = 23501 (0.424 sec)
INFO:tensorflow:global_step/sec: 240.323
INFO:tensorflow:loss = 42.5395, step = 23601 (0.417 sec)
INFO:tensorflow:global_step/sec: 229.379
INFO:tensorflow:loss = 62.4304, step = 23701 (0.435 sec)
INFO:tensorflow:global_step/sec: 233.831
INFO:tensorflow:loss = 56.8523, step = 23801 (0.428 sec)
INFO:tensorflow:global_step/sec: 236.336
INFO:tensorflow:loss = 62.437, step = 23901 (0.422 sec)
INFO:tensorflow:Saving checkpoints for 24000 into trained_models/class-model-01/model.ckpt.
INFO:tensorflow:Loss for final step: 54.2932.

* data input_fn:
================
Input file(s): data/valid-*.tfrecords
Batch size: 500
Epoch Count: 1
Mode: eval
Shuffle: False
================

WARNING:tensorflow:Casting <dtype: 'float32'> labels to bool.
WARNING:tensorflow:Casting <dtype: 'float32'> labels to bool.
INFO:tensorflow:Starting evaluation at 2017-11-22-18:35:49
INFO:tensorflow:Restoring parameters from trained_models/class-model-01/model.ckpt-24000
INFO:tensorflow:Finished evaluation at 2017-11-22-18:35:50
INFO:tensorflow:Saving dict for global step 24000: accuracy = 0.952333, accuracy_baseline = 0.5, auc = 0.991163, auc_precision_recall = 0.991653, average_loss = 0.121782, global_step = 24000, label/mean = 0.5, loss = 60.8909, prediction/mean = 0.497856
INFO:tensorflow:Restoring parameters from trained_models/class-model-01/model.ckpt-24000
INFO:tensorflow:Assets added to graph.
INFO:tensorflow:No assets to write.
INFO:tensorflow:SavedModel written to: b"trained_models/class-model-01/export/predict/temp-b'1511375753'/saved_model.pbtxt"
.......................................
Experiment finished at 18:35:55

Experiment elapsed time: 136.959342 seconds

6. Evaluate the Model





In [14]:



    


TRAIN_SIZE = 12000
VALID_SIZE = 3000
TEST_SIZE = 5000
train_input_fn = lambda: tfrecods_input_fn(files_name_pattern= TRAIN_DATA_FILES_PATTERN, 
                                      mode= tf.estimator.ModeKeys.EVAL,
                                      batch_size= TRAIN_SIZE)

valid_input_fn = lambda: tfrecods_input_fn(files_name_pattern= VALID_DATA_FILES_PATTERN, 
                                      mode= tf.estimator.ModeKeys.EVAL,
                                      batch_size= VALID_SIZE)

test_input_fn = lambda: tfrecods_input_fn(files_name_pattern= TEST_DATA_FILES_PATTERN, 
                                      mode= tf.estimator.ModeKeys.EVAL,
                                      batch_size= TEST_SIZE)

estimator = create_estimator(run_config, hparams)

train_results = estimator.evaluate(input_fn=train_input_fn, steps=1)
print()
print("############################################################################################")
print("# Train Measures: {}".format(train_results))
print("############################################################################################")

valid_results = estimator.evaluate(input_fn=valid_input_fn, steps=1)
print()
print("############################################################################################")
print("# Valid Measures: {}".format(valid_results))
print("############################################################################################")

test_results = estimator.evaluate(input_fn=test_input_fn, steps=1)
print()
print("############################################################################################")
print("# Test Measures: {}".format(test_results))
print("############################################################################################")



    


















    






INFO:tensorflow:Using config: {'_task_type': None, '_task_id': 0, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x117584b70>, '_master': '', '_num_ps_replicas': 0, '_num_worker_replicas': 0, '_environment': 'local', '_is_chief': True, '_evaluation_master': '', '_tf_config': gpu_options {
  per_process_gpu_memory_fraction: 1
}
, '_tf_random_seed': 19830610, '_save_summary_steps': 100, '_save_checkpoints_secs': None, '_log_step_count_steps': 100, '_session_config': None, '_save_checkpoints_steps': 2400, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_model_dir': 'trained_models/class-model-01'}

* data input_fn:
================
Input file(s): data/train-*.tfrecords
Batch size: 12000
Epoch Count: None
Mode: eval
Shuffle: False
================

WARNING:tensorflow:Casting <dtype: 'float32'> labels to bool.
WARNING:tensorflow:Casting <dtype: 'float32'> labels to bool.
INFO:tensorflow:Starting evaluation at 2017-11-22-18:35:57
INFO:tensorflow:Restoring parameters from trained_models/class-model-01/model.ckpt-24000
INFO:tensorflow:Evaluation [1/1]
INFO:tensorflow:Finished evaluation at 2017-11-22-18:35:58
INFO:tensorflow:Saving dict for global step 24000: accuracy = 0.957917, accuracy_baseline = 0.5, auc = 0.993336, auc_precision_recall = 0.993439, average_loss = 0.104192, global_step = 24000, label/mean = 0.5, loss = 1250.3, prediction/mean = 0.502052

############################################################################################
# Train Measures: {'accuracy': 0.95791668, 'accuracy_baseline': 0.5, 'auc': 0.9933362, 'auc_precision_recall': 0.9934386, 'average_loss': 0.10419194, 'label/mean': 0.5, 'loss': 1250.3032, 'prediction/mean': 0.50205207, 'global_step': 24000}
############################################################################################

* data input_fn:
================
Input file(s): data/valid-*.tfrecords
Batch size: 3000
Epoch Count: None
Mode: eval
Shuffle: False
================

WARNING:tensorflow:Casting <dtype: 'float32'> labels to bool.
WARNING:tensorflow:Casting <dtype: 'float32'> labels to bool.
INFO:tensorflow:Starting evaluation at 2017-11-22-18:36:00
INFO:tensorflow:Restoring parameters from trained_models/class-model-01/model.ckpt-24000
INFO:tensorflow:Evaluation [1/1]
INFO:tensorflow:Finished evaluation at 2017-11-22-18:36:02
INFO:tensorflow:Saving dict for global step 24000: accuracy = 0.952333, accuracy_baseline = 0.5, auc = 0.991163, auc_precision_recall = 0.991653, average_loss = 0.121782, global_step = 24000, label/mean = 0.5, loss = 365.345, prediction/mean = 0.497856

############################################################################################
# Valid Measures: {'accuracy': 0.95233333, 'accuracy_baseline': 0.5, 'auc': 0.99116331, 'auc_precision_recall': 0.99165344, 'average_loss': 0.12178176, 'label/mean': 0.5, 'loss': 365.34528, 'prediction/mean': 0.49785584, 'global_step': 24000}
############################################################################################

* data input_fn:
================
Input file(s): data/test-*.tfrecords
Batch size: 5000
Epoch Count: None
Mode: eval
Shuffle: False
================

WARNING:tensorflow:Casting <dtype: 'float32'> labels to bool.
WARNING:tensorflow:Casting <dtype: 'float32'> labels to bool.
INFO:tensorflow:Starting evaluation at 2017-11-22-18:36:03
INFO:tensorflow:Restoring parameters from trained_models/class-model-01/model.ckpt-24000
INFO:tensorflow:Evaluation [1/1]
INFO:tensorflow:Finished evaluation at 2017-11-22-18:36:05
INFO:tensorflow:Saving dict for global step 24000: accuracy = 0.9524, accuracy_baseline = 0.5, auc = 0.990539, auc_precision_recall = 0.991224, average_loss = 0.123451, global_step = 24000, label/mean = 0.5, loss = 617.257, prediction/mean = 0.499188

############################################################################################
# Test Measures: {'accuracy': 0.95240003, 'accuracy_baseline': 0.5, 'auc': 0.99053907, 'auc_precision_recall': 0.99122417, 'average_loss': 0.12345144, 'label/mean': 0.5, 'loss': 617.2572, 'prediction/mean': 0.49918783, 'global_step': 24000}
############################################################################################

7. Prediction





In [15]:



    


import itertools

predict_input_fn = lambda: tfrecods_input_fn(files_name_pattern= TEST_DATA_FILES_PATTERN, 
                                      mode= tf.estimator.ModeKeys.PREDICT,
                                      batch_size= 5)


predictions = list(itertools.islice(estimator.predict(input_fn=predict_input_fn),5))

print("")
print("* Predicted Classes: {}".format(list(map(lambda item: item["class_ids"][0]
    ,predictions))))

print("* Predicted Probabilities: {}".format(list(map(lambda item: list(item["probabilities"])
    ,predictions))))



    


















    






* data input_fn:
================
Input file(s): data/test-*.tfrecords
Batch size: 5
Epoch Count: None
Mode: infer
Shuffle: False
================

WARNING:tensorflow:Input graph does not contain a QueueRunner. That means predict yields forever. This is probably a mistake.
INFO:tensorflow:Restoring parameters from trained_models/class-model-01/model.ckpt-24000

* Predicted Classes: [0, 1, 1, 0, 1]
* Predicted Probabilities: [[0.98284829, 0.017151766], [0.00044470496, 0.99955529], [0.00017617996, 0.99982387], [0.99743325, 0.0025667532], [0.010059897, 0.98994011]]

Serving via the Saved Model





In [16]:



    


import os

export_dir = model_dir +"/export/predict/"

saved_model_dir = export_dir + "/" + os.listdir(path=export_dir)[-1] 

print(saved_model_dir)

predictor_fn = tf.contrib.predictor.from_saved_model(
    export_dir = saved_model_dir,
    signature_def_key="predict"
)

output = predictor_fn({'csv_rows': ["0.5,1,ax01,bx02", "-0.5,-1,ax02,bx02"]})
print(output)



    


















    






trained_models/class-model-01/export/predict//1511375753
INFO:tensorflow:Restoring parameters from b'trained_models/class-model-01/export/predict//1511375753/variables/variables'
{'class_ids': array([[1],
       [1]]), 'classes': array([[b'negative'],
       [b'negative']], dtype=object), 'logistic': array([[ 0.99826616],
       [ 0.99829525]], dtype=float32), 'logits': array([[ 6.35565805],
       [ 6.37263775]], dtype=float32), 'probabilities': array([[ 0.00173388,  0.99826616],
       [ 0.00170474,  0.99829525]], dtype=float32)}

















In [ ]:

Content source: GoogleCloudPlatform/tf-estimator-tutorials

Similar notebooks:

notebook.community | gallery | about